#CHAPTER 5. Discovering the World

Throughout this book, we talk about many varieties of XMPP entities: servers, clients,bots, chat rooms, pubsub nodes, etc. On the public XMPP network, all of these entities come in multiple flavors. For instance, there are at least half a dozen popular XMPP server implementations, and many more XMPP clients for just about every device and operating system. Furthermore, there are hundreds of possible features that an XMPP entity can support, including standardized protocols (to which the XMPP Standards Foundation is always adding), user-configurable options, client plug-ins, server modules, and more. And let’s not forget that many of these software projects are quite active,frequently releasing updated versions.
This diversity is tremendously powerful, but it raises two important questions:
1. How can you learn what entities are out there on the network?
2. Once you find them, how can you determine which XMPP features they support?
To answer these questions, you need service discovery (often called “disco” by XMPP developers).
When might you want to use service discovery? You might be learning a foreign language, so you want to find a chat room where you can practice; you might be interested in using a specific publish-subscribe mechanism, so you want to discover a pubsub service where that mechanism is supported; you might like to figure out whether one of your friends or colleagues has video chat capabilities; you might even want to announce to all your contacts that you’re interested in finding out what music they’re listening to. All of these tasks (and more) can be completed using the techniques discussed in this chapter.

##Items and Info

The XMPP service discovery protocol defined in [XEP-0030] provides two basic discovery methods. The first, known as disco#items , enables you to discover entities. The second, known as disco#info , enables you to discover which features a given entity supports. Let’s look at each of these in turn.
It does you no good to discover features unless you have first discovered some entities. A client always knows about at least one entity: the server it connects to. And since XMPP servers typically host additional entities such as pubsub topics and multi-user chat rooms, clients often need to discover those additional entities. Such discovery happens using the disco#items half of the XMPP service discovery protocol by sending an IQ-get to the server. Here, the Mad Hatter queries the wonderland.lit server:

><iq from="hatter@wonderland.lit/home"
>id="xl391n47"
>to="wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#items"/>
></iq>

That command means “please send me all the items that are associated with wonderland.lit ; the server then replies with a list of associated entities, which the client tracks by the value of the id attribute:

><iq from="wonderland.lit"
>id="xl391n47"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#items">
><item jid="conference.wonderland.lit"/>
><item jid="notify.wonderland.lit"/>
></query>
></iq>

The only two associated entities in this case are conference.wonderland.lit and notify.wonderland.lit . But what are these entities? What features do they support? To find out, the Mad Hatter needs to query each one individually using the disco#info method. Here the Mad Hatter queries the conference.wonderland.lit service:

><iq from="hatter@wonderland.lit/home"
>id="gq02kb71"
>to="conference.wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"/>
></iq>

The XML element for both the items request and the info request is <query/> , but the requests are differentiated by the XML namespace: the items request is qualified by the http://jabber.org/protocol/disco#items namespace, whereas the info request is qualified by the http://jabber.org/protocol/disco#info namespace.
Now, the conference.wonderland.lit service returns some information about itself. (Web developers can think of this as similar to the results returned in the HTTP Accept , Accept-Charset , Accept-Encoding , Accept-Language , and Accept-Ranges response headers, except that the disco#info response is more extensible.)

><iq from="conference.wonderland.lit"
>id="gq02kb71"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info">
><identity category="conference" type="text" name="Chatrooms"/>
><feature var="http://jabber.org/protocol/muc"/>
><feature var="jabber:iq:register"/>
><feature var="vcard-temp"/>
></query>
></iq>

By interpreting the XML in the foregoing example, the Mad Hatter learns that wonderland.lit hosts a service conference.wonderland.lit , which provides a text conferencing service that supports the Multi-User Chat protocol (the http://jabber.org/protocol/muc namespace defined in [XEP-0045]), in-band registration of usernames (the jabber:iq:register namespace defined in [XEP-0077]), and component vCards
(the vcard-temp namespace defined in [XEP-0054]).


----------What’s Your Identity?----------

In most uses of service discovery, we’re mainly interested in the particular features that another entity supports. The <identity/> element provides a more general clue about what kind of entity this is. The first discovery protocol used in the XMPP community, called “Agent Information” (see the historical [XEP-0094]), did not disclose detailed features, but did advertise basic identities, which is why the modern service discovery protocol includes identities as well. (In fact, an entity can advertise multiple identities at the same time, such as a groupchat service that is simultaneously a native XMPP Multi-User Chat service and a gateway to Internet Relay Chat.)

--------------------


##Using Service Discovery with Servers and Services

The disco#items and disco#info methods are typically used together to “walk the tree” of entities. Consider a typical sequence:
1. Send a disco#items query to the wonderland.lit server, discovering (among others) the conference.wonderland.lit service.
2. Send a disco#info query to the conference.wonderland.lit room, in order to discover that the conference.wonderland.lit service is a multi-user chat service.
3. Send a disco#items query to the conference.wonderland.lit service, discovering (among others) the tea@conference.wonderland.lit room.
4. Send a disco#info query to the tea@conference.wonderland.lit room to find out more information about the room (e.g., its name, natural language, and other configuration options).
5. Send a disco#items query to the tea@conference.wonderland.lit service, discovering the tea@conference.wonderland.lit/alice user.
Clearly, quite a bit of back-and-forth is needed to generate a complete picture of an entity hierarchy. In common usage, a client would not walk the entire tree automatically. Instead, it would not go beyond the first few queries unless the user requests detailed information about, say, a particular chat room and its users.
What follows is the full sequence just outlined.
First, query the wonderland.lit server for its associated items:

><iq from="hatter@wonderland.lit/home"
>id="ris71b37"
>to="wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#items"/>
></iq>
><iq from="wonderland.lit"
>id="ris71b37"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#items">
><item jid="conference.wonderland.lit"/>
><item jid="notify.wonderland.lit"/>
></query>
></iq>

Second, query the conference.wonderland.lit service to see what kind of entity it is:

><iq from="hatter@wonderland.lit/home"
>id="hs82bd67"
>to="conference.wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"/>
></iq>
><iq from="conference.wonderland.lit"
>id="hs82bd67"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info">
><identity category="conference" type="text" name="Chatrooms"/>
><feature var="http://jabber.org/protocol/muc"/>
><feature var="jabber:iq:register"/>
><feature var="vcard-temp"/>
></query>
></iq>

Third, query the conference.wonderland.lit service for its associated items:

><iq from="hatter@wonderland.lit/home"
>id="skf81ga8"
>to="conference.wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#items"/>
></iq>
><iq from="conference.wonderland.lit"
>id="skf81ga8"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#items">
><item jid="pool@conference.wonderland.lit"/>
><item jid="tea@conference.wonderland.lit"/>
></query>
></iq>

Fourth, query a particular conference room for its information:

><iq from="hatter@wonderland.lit/home"
>id="ow8x71b6"
>to="tea@conference.wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"/>
></iq>

The room then returns a list of its configured features, such as the fact that it is public (anyone can discover it), persistent (it won’t go away when the last person leaves), open (anyone can join it), semi-anonymous (only the room admins can find out your real JabberID), unmoderated (new users have voice), and unsecured (no password is required to join). Here, the room also provides additional information about itself by including a data form of the kind we discuss in Chapter 6 (this extension mechanism for service discovery is defined in [XEP-0128]):

><iq from="tea@conference.wonderland.lit"
>id="ow8x71b6"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info">
><identity category="conference" type="text" name="The Tea Room"/>
><feature var="http://jabber.org/protocol/muc"/>
><feature var="muc_public"/>
><feature var="muc_persistent"/>
><feature var="muc_open"/>
><feature var="muc_semianonymous"/>
><feature var="muc_unmoderated"/>
><x xmlns="jabber:x:data" type="result">
><field type="hidden" var="FORM_TYPE">
><value>http://jabber.org/protocol/muc#roominfo</value>
></field>
><field label="Number of occupants" var="muc#roominfo_occupants">
><value>4</value>
></field>
></x>
></query>
></iq>

Furthermore, querying a conference room with the disco#items namespace may return a list of JIDs in the room, as shown in the following examples:

><iq from="hatter@wonderland.lit/home"
>id="ac4cf"
>to="tea@conference.wonderland.lit"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#items"/>
></iq>
><iq from="tea@conference.wonderland.lit"
>id="ac4cf"
>to="hatter@wonderland.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#items">
><item name="Alice" jid="tea@conference.wonderland.lit/Alice"/>
><item name="Hare" jid="tea@conference.wonderland.lit/Hare"/>
><item name="Hatter" jid="tea@conference.wonderland.lit/Hatter"/>
><item name="Dormouse" jid="tea@conference.wonderland.lit/Dormouse"/>
></query>
></iq>

##Using Service Discovery with Clients

When it comes to discovering the capabilities of other clients on the network, there are two tools at your disposal: explicit service discovery of the kind we’ve already discussed, and a kind of service discovery shorthand that is advertised in XMPP presence notifications. We’ll look at each of these in turn.

###Explicit Service Discovery

In the last section, we said that a client always knows about at least one entity: its server. Yet we’ve also seen that usually a client knows about some other entities: the items in its roster. What can those entities do?
Presence plays an important part in helping us find out. As you may recall from Chapter 3, when a client goes online, its server sends presence probes to each of the user’s contacts. The server for each contact then returns information about the available resources for that contact. This information is not a service discovery list, but a series of presence stanzas, such as the following presence notifications that Alice’s sister receives when she logs in:

><presence from="alice@wonderland.lit/rabbithole"
>to="sister@realworld.lit"/>
><presence from="alice@wonderland.lit/party"
>to="sister@realworld.lit"/>
><presence from="friend@school.lit/laptop"
>to="sister@realworld.lit"/>


----------Disco and Presence----------

Presence helps here because in order to exchange IQ stanzas with another user, you need to know the person’s full JID ( user@domain.tld/resource ). Because presence notifications come from the full JID, they are essentially a kind of push format for the data you would need to poll for via the disco#items namespace for each of your contacts.

--------------------



Now let’s say that Alice’s sister wants to find out what each of Alice’s devices can do (this kind of information can be used to populate a drop-down box of possible actions, such as “send a file” or “start voice chat”). To find out, she sends a disco#info request to each of Alice’s resources:

><iq from="sister@realworld.lit/home"
>id="p982bs61"
>to="alice@wonderland.lit/rabbithole"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"/>
></iq>
><iq from="sister@realworld.lit/home"
>id="sc374g15"
>to="alice@wonderland.lit/party"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"/>
></iq>

For each of Alice’s resources, she receives a reply telling her what features are supported:

><iq from="alice@wonderland.lit/rabbithole"
>id="p982bs61"
>to="sister@realworld.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info">
><identity category="client" name="Exodus 0.9.1" type="pc"/>
><feature var="http://jabber.org/protocol/caps"/>
><feature var="http://jabber.org/protocol/disco#info"/>
><feature var="http://jabber.org/protocol/disco#items"/>
><feature var="http://jabber.org/protocol/muc"/>
><feature var="jabber:iq:version"/>
></query>
></iq>

><iq from="alice@wonderland.lit/party"
>id="sc374g15"
>to="sister@realworld.lit/home"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info">
><identity category="client" name="Psi" type="pc"/>
><feature var="http://jabber.org/protocol/bytestreams"/>
><feature var="http://jabber.org/protocol/si"/>
><feature var="http://jabber.org/protocol/si/profile/file-transfer"/>
><feature var="http://jabber.org/protocol/disco#info"/>
><feature var="http://jabber.org/protocol/commands"/>
><feature var="http://jabber.org/protocol/rosterx"/>
><feature var="http://jabber.org/protocol/muc"/>
><feature var="jabber:x:data"/>
><feature var="http://jabber.org/protocol/chatstates"/>
><feature var="http://jabber.org/protocol/mood+notify"/>
><feature var="http://jabber.org/protocol/tune+notify"/>
><feature var="http://jabber.org/protocol/physloc+notify"/>
><feature var="http://jabber.org/protocol/geoloc+notify"/>
><feature var="http://www.xmpp.org/extensions/xep-0084.html#ns-metadata+notify"/>
><feature var="http://jabber.org/protocol/xhtml-im"/>
><feature var="urn:xmpp:tmp:sxe"/>
><feature var="http://www.w3.org/2000/svg"/>
></query>
></iq>

That’s a lot of namespaces! Imagine if you sent a disco#info request to everyone in your roster, and you had 500 or 1,000 or 2,000 contacts. You’re probably thinking, “Isn’t all that traffic going to get expensive? There must be a better way!” And there is. We discuss it in the next section.

###Entity Capabilities: Service Discovery Shorthand

The XMPP community has developed an optimized protocol for discovering supported features, at least when you share presence (which you typically do with people in your roster). This Entity Capabilities [XEP-0115] protocol uses presence as the transport for a kind of shorthand service discovery notation. (In fact, just as presence itself is a specialized push version of disco#items data, the entity capabilities protocol uses the XMPP presence “transport” to push out a shorthand notation for disco#info data.)
Entity Capabilities is based on the observation that many of the entities with which you interact will be running the same software, and that this software changes relatively infrequently (that is, if you see version 0.13 of the Psi client today, there’s a good chance you’ll see it again tomorrow). Under this scheme, entities advertise the features they support in their presence stanzas. However, it wouldn’t save any bandwidth to send along the full list of features. Therefore, the features are concatenated and hashed into a short verification code that uniquely identifies the current feature set. The first time your client sees this code, it sends a standard disco#info request to find out which features correspond to the code. The good thing is that whenever your client sees this code in the future, it automatically knows exactly which features the person supports, so no disco#info requests are needed, thus saving a lot of bandwidth (especially on login).
ow does this work in practice? When Alice logs in, her client concatenates and hashes all of the features it supports and then attaches a <c/> child to the presence packet containing the verification code in the ver attribute:

><presence from="alice@wonderland.lit/rabbithole">
><c xmlns="http://jabber.org/protocol/caps"
>hash="sha-1"
>node="http://code.google.com/p/exodus"
>er="QgayPKawpkPSDYmwT/WM94uAlu0="/>
></presence>

The hash attribute identifies the hashing algorithm used (the default is SHA-1). 
The node attribute identifies the software application in use by means of a URI that
provides further information about the codebase.
Both Alice’s sister and the White Rabbit receive Alice’s presence notification. Alice’s sister has received this verification code before and her client has remembered the capabilities associated with it, so she doesn’t need to send a disco#info request to Alice.
However, this is the first time that the White Rabbit’s client has seen this verification code, and so it sends a disco#info request to Alice’s client for the full capabilities:

><iq from="rabbit@wonderland.lit/pool"
>id="v584fak2"
>to="alice@wonderland.lit/rabbithole"
>type="get">
><query xmlns="http://jabber.org/protocol/disco#info"
>node="http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0="/>
></iq>

This query can be recognized as a standard service discovery query, with an extra node attribute specified, constructed from the node and ver attributes of the <c/> element. The reason for the node specification, rather than using a disco query without the node attribute, is to prevent the possible race condition where the White Rabbit’s client “discos” Alice, but Alice’s client has changed its capabilities before receiving the disco request, replying with a set of capabilities that do not match the verification code.
Alice replies to the disco#info request in the usual way:

><iq from="alice@wonderland.lit/rabbithole"
>id="v584fak2"
>to="rabbit@wonderland.lit/pool"
>type="result">
><query xmlns="http://jabber.org/protocol/disco#info"
>node="http://code.google.com/p/exodus#QgayPKawpkPSDYmwT/WM94uAlu0=">
><identity category="client" name="Exodus 0.9.1" type="pc"/>
><feature var="http://jabber.org/protocol/caps"/>
><feature var="http://jabber.org/protocol/disco#info"/>
><feature var="http://jabber.org/protocol/disco#items"/>
><feature var="http://jabber.org/protocol/muc"/>
></query>
></iq>

Upon receiving the response from Alice’s client, the White Rabbit’s client can cache the result, remembering that future clients seen with the same <c/> element support the features that Alice has advertised. The result: no need to send a flood of service discovery requests when you log in. (The XMPP network thanks you!)
In Chapter 8, we’ll see how entity capabilities are used to optimize the processes of discovering and subscribing to real-time information sources over the XMPP network.

##Summary

Service discovery is a key building block of XMPP-based services because it is extremely helpful to determine what entities are out there on the network and exactly which XMPP features those entities implement. Using only the ability to find other entities ( disco#items ) and to query their capabilities ( disco#info ), you can discover a wealth of services on the network and discover how those services and the entities in your roster can interact with you in real time.






